{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Siamese networks with TensorFlow 2.0/Keras\n",
    "\n",
    "In this example, we'll implement a simple siamese network system, which verifyies whether a pair of MNIST images is of the same class (true) or not (false). \n",
    "\n",
    "_This example is partially based on_ [https://github.com/keras-team/keras/blob/master/examples/mnist_siamese.py](https://github.com/keras-team/keras/blob/master/examples/mnist_siamese.py)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's start with the imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "\n",
    "import numpy as np\n",
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll continue with the `create_pairs` function, which creates a training dataset of equal number of true/false pairs of each MNIST class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_pairs(inputs: np.ndarray, labels: np.ndarray):\n",
    "    \"\"\"Create equal number of true/false pairs of samples\"\"\"\n",
    "\n",
    "    num_classes = 10\n",
    "\n",
    "    digit_indices = [np.where(labels == i)[0] for i in range(num_classes)]\n",
    "    pairs = list()\n",
    "    labels = list()\n",
    "    n = min([len(digit_indices[d]) for d in range(num_classes)]) - 1\n",
    "    for d in range(num_classes):\n",
    "        for i in range(n):\n",
    "            z1, z2 = digit_indices[d][i], digit_indices[d][i + 1]\n",
    "            pairs += [[inputs[z1], inputs[z2]]]\n",
    "            inc = random.randrange(1, num_classes)\n",
    "            dn = (d + inc) % num_classes\n",
    "            z1, z2 = digit_indices[d][i], digit_indices[dn][i]\n",
    "            pairs += [[inputs[z1], inputs[z2]]]\n",
    "            labels += [1, 0]\n",
    "    return np.array(pairs), np.array(labels, dtype=np.float32)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we'll define the base network of the siamese system:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_base_network():\n",
    "    \"\"\"The shared encoding part of the siamese network\"\"\"\n",
    "\n",
    "    return tf.keras.models.Sequential([\n",
    "        tf.keras.layers.Flatten(),\n",
    "        tf.keras.layers.Dense(128, activation='relu'),\n",
    "        tf.keras.layers.Dropout(0.1),\n",
    "        tf.keras.layers.Dense(128, activation='relu'),\n",
    "        tf.keras.layers.Dropout(0.1),\n",
    "        tf.keras.layers.Dense(64, activation='relu'),\n",
    "    ])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, let's load the regular MNIST training and validation sets and create true/false pairs out of them:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load the train and test MNIST datasets\n",
    "(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()\n",
    "x_train = x_train.astype(np.float32)\n",
    "x_test = x_test.astype(np.float32)\n",
    "x_train /= 255\n",
    "x_test /= 255\n",
    "input_shape = x_train.shape[1:]\n",
    "\n",
    "# Create true/false training and testing pairs\n",
    "train_pairs, tr_labels = create_pairs(x_train, y_train)\n",
    "test_pairs, test_labels = create_pairs(x_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then, we'll build the siamese system, which includes the `base_network`, the 2 siamese paths `encoder_a` and `encoder_b`, the `l1_dist` measure, and the combined `model`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create the siamese network\n",
    "# Start from the shared layers\n",
    "base_network = create_base_network()\n",
    "\n",
    "# Create first half of the siamese system\n",
    "input_a = tf.keras.layers.Input(shape=input_shape)\n",
    "\n",
    "# Note how we reuse the base_network in both halfs\n",
    "encoder_a = base_network(input_a)\n",
    "\n",
    "# Create the second half of the siamese system\n",
    "input_b = tf.keras.layers.Input(shape=input_shape)\n",
    "encoder_b = base_network(input_b)\n",
    "\n",
    "# Create the the distance measure\n",
    "l1_dist = tf.keras.layers.Lambda(\n",
    "    lambda embeddings: tf.keras.backend.abs(embeddings[0] - embeddings[1])) \\\n",
    "    ([encoder_a, encoder_b])\n",
    "\n",
    "# Final fc layer with a single logistic output for the binary classification\n",
    "flattened_weighted_distance = tf.keras.layers.Dense(1, activation='sigmoid') \\\n",
    "    (l1_dist)\n",
    "\n",
    "# Build the model\n",
    "model = tf.keras.models.Model([input_a, input_b], flattened_weighted_distance)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we can train the model and check the validation accuracy, which reaches 99.37%:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 108400 samples, validate on 17820 samples\n",
      "Epoch 1/20\n",
      "108400/108400 [==============================] - 5s 44us/sample - loss: 0.3328 - accuracy: 0.8540 - val_loss: 0.2435 - val_accuracy: 0.9184\n",
      "Epoch 2/20\n",
      "108400/108400 [==============================] - 4s 37us/sample - loss: 0.1612 - accuracy: 0.9409 - val_loss: 0.1672 - val_accuracy: 0.9465\n",
      "Epoch 3/20\n",
      "108400/108400 [==============================] - 4s 38us/sample - loss: 0.1096 - accuracy: 0.9611 - val_loss: 0.1221 - val_accuracy: 0.9625\n",
      "Epoch 4/20\n",
      "108400/108400 [==============================] - 4s 38us/sample - loss: 0.0824 - accuracy: 0.9712 - val_loss: 0.1052 - val_accuracy: 0.9667\n",
      "Epoch 5/20\n",
      "108400/108400 [==============================] - 4s 37us/sample - loss: 0.0673 - accuracy: 0.9760 - val_loss: 0.0958 - val_accuracy: 0.9706\n",
      "Epoch 6/20\n",
      "108400/108400 [==============================] - 4s 37us/sample - loss: 0.0542 - accuracy: 0.9808 - val_loss: 0.1054 - val_accuracy: 0.9689\n",
      "Epoch 7/20\n",
      "108400/108400 [==============================] - 4s 37us/sample - loss: 0.0471 - accuracy: 0.9832 - val_loss: 0.0823 - val_accuracy: 0.9764\n",
      "Epoch 8/20\n",
      "108400/108400 [==============================] - 4s 37us/sample - loss: 0.0410 - accuracy: 0.9853 - val_loss: 0.0769 - val_accuracy: 0.9769\n",
      "Epoch 9/20\n",
      "108400/108400 [==============================] - 4s 37us/sample - loss: 0.0377 - accuracy: 0.9868 - val_loss: 0.0921 - val_accuracy: 0.9731\n",
      "Epoch 10/20\n",
      "108400/108400 [==============================] - 4s 37us/sample - loss: 0.0326 - accuracy: 0.9887 - val_loss: 0.0920 - val_accuracy: 0.9744\n",
      "Epoch 11/20\n",
      "108400/108400 [==============================] - 4s 37us/sample - loss: 0.0309 - accuracy: 0.9887 - val_loss: 0.0846 - val_accuracy: 0.9753\n",
      "Epoch 12/20\n",
      "108400/108400 [==============================] - 4s 37us/sample - loss: 0.0283 - accuracy: 0.9898 - val_loss: 0.0902 - val_accuracy: 0.9742\n",
      "Epoch 13/20\n",
      "108400/108400 [==============================] - 4s 37us/sample - loss: 0.0262 - accuracy: 0.9908 - val_loss: 0.0956 - val_accuracy: 0.9753\n",
      "Epoch 14/20\n",
      "108400/108400 [==============================] - 4s 37us/sample - loss: 0.0228 - accuracy: 0.9918 - val_loss: 0.0820 - val_accuracy: 0.9781\n",
      "Epoch 15/20\n",
      "108400/108400 [==============================] - 4s 37us/sample - loss: 0.0240 - accuracy: 0.9914 - val_loss: 0.0869 - val_accuracy: 0.9759\n",
      "Epoch 16/20\n",
      "108400/108400 [==============================] - 4s 37us/sample - loss: 0.0225 - accuracy: 0.9921 - val_loss: 0.0754 - val_accuracy: 0.9794\n",
      "Epoch 17/20\n",
      "108400/108400 [==============================] - 4s 37us/sample - loss: 0.0209 - accuracy: 0.9928 - val_loss: 0.0786 - val_accuracy: 0.9778\n",
      "Epoch 18/20\n",
      "108400/108400 [==============================] - 4s 37us/sample - loss: 0.0207 - accuracy: 0.9929 - val_loss: 0.0797 - val_accuracy: 0.9787\n",
      "Epoch 19/20\n",
      "108400/108400 [==============================] - 4s 37us/sample - loss: 0.0178 - accuracy: 0.9937 - val_loss: 0.0884 - val_accuracy: 0.9785\n",
      "Epoch 20/20\n",
      "108400/108400 [==============================] - 4s 37us/sample - loss: 0.0189 - accuracy: 0.9935 - val_loss: 0.0754 - val_accuracy: 0.9799\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7f455027b4a8>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Train\n",
    "model.compile(loss='binary_crossentropy',\n",
    "              optimizer=tf.keras.optimizers.Adam(),\n",
    "              metrics=['accuracy'])\n",
    "\n",
    "model.fit([train_pairs[:, 0], train_pairs[:, 1]], tr_labels,\n",
    "          batch_size=128,\n",
    "          epochs=20,\n",
    "          validation_data=([test_pairs[:, 0], test_pairs[:, 1]], test_labels))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
